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Abstract 

A delayed differential equation modelling a single neuron with inertial term is considered in this 
paper. Hopf bifurcation is studied by using the normal form theory of retarded functional differential 
equations. When adopting a nonmonotonic activation function, chaotic behavior is observed. Phase 
plots, waveform plots, and power spectra are presented to confirm the chaoticity. 
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1 Introduction 

In recent years, dynamical characteristics of neural networks have become a focal subject of intensive 
research studies. Bifurcations and chaotic phenomena have been investigated in various neural networks. 
For example, chaotic solutions were obtained in a neural network consisting of 26 neurons in [1]. Nu- 
merical solutions of differential equations with electronic circuit models of chaotic neural networks were 
qualitatively studied in [2]. In [3], a chaotic neural network with four neurons was investigated. Chaotic 
behavior was found in a cellular neural network with three cells in [4]. In [5], chaotic phenomenon in a 
three-neuron hysteretic Hopficld-type neural network was discussed. In [6], a high-dimensional chaotic 
neural network under external sinusoidal force was studied. In [7], bifurcation and chaos as well as their 
control in a system of strongly connected neural oscillators were discussed. In [8, 9], a discrete-time 
transiently chaotic neural network was studied. The chaotic phenomenon in a neural network learning 
algorithm was reported in [10]. Moreover, chaotic behaviors of inertial neural networks are studied in 
[11, 12]. On the other hand, there are extensive literatures studied neural network models with delays. 
For example, bifurcations and chaotic dynamics of neural networks with discrete and distributed delays 
were studied in [13-21]. 
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In this paper, the dynamical behaviors of a single delayed neuron model with inertial terms are inves- 
tigated. The work presented in this paper can be considered as an extension of the works for inertial 
neural network without delays [11, 12] to the case with delays, or an extension of the work for single 
neuron without inertial terms [13] to the case with inertial terms. 

The paper is organized as follows. The delayed inertial neuron model is described, and the local 
stability and the existence of Hopf bifurcation is studied in Section 2. In Section 3, the properties of 
the bifurcating periodic solutions are analyzed based on the normal form theory developed in [22]. To 
justify the theoretical analysis, a numerical example is given in Section 4. In Section 5, the observed 
chaotic behavior of the model with a nonmonotonic activation function is reported. Finally, conclusions 
are drawn in Section 6. 



2 Local stability and the existence of Hopf bifurcation 

The single inertial neuron with time delay, similar to that in [13] but with an inertial term, is described 

by 

x = —ax — bx + cf(x — hx(t — r)) (1) 

where constants a, b, c > 0, h > 0, and t > is the time delay. Without loss of generality, assume that 
the activation function /(.) in the above equation is a nonlinear function and its third-order continuous 
derivative exists. Define 

Xi(t) = x(t) — hx(t — t), t E [— r, +oo) 

Then, we have 

±i(t) =x 2 (t) 

x 2 (t) = -ax 2 (t) - bxi(t) + cf(x 1 (t)) - chf(xi(t - r)) 

The phase space is C := C([—t, 0]; i? 2 ). Throughout this paper, assume that the following conditions are 
satisfied: 

/(0) = 0, /'(0) > 0, \c(l-h)\<l/f'(0) 

It is clear that (2) has an unique equilibrium (0, 0) under the above condition. It is also easy to see that 
if there is no delay term in (1), i.e. h = 0, then the model is asymptotically stable when 

6-c/'(0)>0. (3) 

In the following, we estimate the value of h that preserves the system stability under the above condition. 

For convenience, we restate here a result of Bellman and Cooke [24, Theorem 13.9]. 
Lemma 1 [24]: Let H{z) = (z 2 + pz + q)e z + r, where p is real and positive, q is real and nonnegative, 
and r is real. Denote by a,k {k > 0) the sole root of the equation 

cot a = (a 2 — q)/ap 

which lies on the interval {kit, kit + ir). Define the natural number n as follows: 
1. if r > and p 2 >2q,n = 1; 
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2. if r > and p 2 < 2q, n is the odd integer k for which cik lies closest to y/q — p 2 /2; 

3. if r < and p 2 >2q,n = 2; 



4. if r < and p 2 < 2q, n is the even integer k for which a k lies closest to y'g — p 2 /2. 

Then, a necessary and sufficient condition under which all the roots of H{z) = lie to the left of the 
imaginary axis is that 

1. r > and (r sina n )/pa n < 1, or 

2. — q < r < and (r sin a n )/pa n < 1. 

Separating the linear and the nonlinear terms, (2) becomes 

x = L(x t ) + F(x t ) 



(4) 



where x t G C, x t (#) = a;(t + 8), — r < 9 < 0, and L : C — > i? 2 , F : C — > i? 2 are given respectively by 



MO) 

-(b - c/'(0))^(0) - a^ 2 (0) - c^/'(0)^i(-t) 



^^(0) + *CffiM(0) - ^M^(-r) - ^^(-r) + 



(5) 



with = ((^i, (^2) G C. Here and throughout this paper, we refer to [25] for notation and classical results 
on functional differential equations (FDEs), including such as Eq. (4). 
The characteristic equation for the linearization of Eq.(4) at (0, 0) is 



Let s = At. Then we have 



A 2 + a\ + (b - c/'(0)) + chf'(0)e- XT = 



[s 2 + ars + (b- c/'(0))r 2 ]e s + c/i/'(0)t 2 = 



(6) 



(7) 



The fixed point is locally stable if all roots of the above equation have negative real parts [25]. For 
each r, we are interested in the maximum value of h such that the system is locally stable. 
Theorem 1: Denote by Wk {k > 0) the sole root of the equation 

COtW = [ W 2 - (b - cf'(Q))T 2 ]/aTW 

which lies on the interval (kn, k-K + ir). Define the nature number n as follows: 
1. if a 2 > 2(6-c/'(0)), n= 1; 



2. if a 2 < 2(6 - c/'(0)), n is the odd k for which w k lies closest to y/(b - c/'(0)) - a 2 /2r 

Then, under condition (3), a necessary and sufficient condition that the solution of (2) is asymptotically 
stable is that 



Proof: Since a > 0, r > 0, /'(0) > 0, b — c/'(0) > 0, h > 0, a direct application of Lemma 1 to (7) with 
p = ar,q = (b — c/'(0))t 2 and r — c/i/'(0)t 2 proves the claim. 

In the following, we study the existence of Hopf bifurcation in Eq. (2) by choosing h as the bifurcation 
parameter. First, we would like to know when Eq. (6) has purely imaginary roots A = ±iui (lu > 0) at 
h = Hq. If A = ±iujQ, u>o > 0, we have 

chf (0)cos lu t =u%-{b- c/'(0)) 
chf (0)sino> ' r = a ^o 



The above equations imply that 



and, consequently, 



ujI - (6-c/'(0)) , , 
cotw T = -2 — ^ -LAJl = 5 ( Wo ) 

aw 



So, g(u>) is strictly monotonically increasing on (0, +oo), with lim^ ->o g(uj) = — oo and lim w ^ +oc g(u>) — 
+oo. Clearly, g(uj) intersects cot wr only at a point. Hence, A = ±«w are simple roots of Eq. (6), where 
uj is the unique root of cotwr = " 2 ~ (fe ~J /,(0)) , and ft = c/ , (0 °^" ncJoT ■ 

From [26], we know that all the other roots of Eq. (6) have negative real parts. We proceed to calculate 
Re[dX/dh] at h = ho. Differentiating Eq. (6) with respect to h yields 

d\ _ c,f(0)e- Ar 

dh ~ chTf'(0)e- XT - 2A - a 

So, we have 

Re 



dX 



dh 



cf'(0)[cf'(0)h O T — acoswoT + 2uj n smLj n T] 



c^o (cftor/'(0)cos ujot — a) 2 + (c/ior/'(0)sina;oT + 2luq) 2 

From the above analysis, we have the following result. 
Theorem 2: Under condition (3), if cf'(0)hoT — acosutoT + 2ojQsmujQT ^ 0, then as h pass through the 
critical value ho = y <( - °^ m 1 there is a Hopf bifurcation of system (1) at its equilibrium (0, 0), where 



u>o is the sole root of cot ujt — 



au) 



Remark: Note that if we let w = loqt, then the constant ho in Theorem 2 can be rewritten as 
i = 7T, V ' — i which is consistent to that in Theorem 1. 



3 Direction and stability of bifurcating periodic solutions 

In this section, we study the direction and stability of the bifurcating periodic solutions. The method 
used here is based on the normal form theory developed by Faria and Magalhaes [22]. This method 
computes normal forms for retarded functional differential equations, without computing beforehand the 
center manifold of the singularity. 

As in [21], in the following we assume /"(0) = 0, /'"(0) ^ 0. Define A = {— iujo,i^o} and introduce 
the new parameter (3 = h — h . Eq. (4) can be rewritten as 

x = L (x t )+F (x u f3) (8) 
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where 



Fo(<P) 



^(0) 



-(6 - c/'(0))^(0) - a^ 2 (0) - c/i /'(0)^i(-r) 



-c/'(0)M(-r) + 2^1^(0) - ^(^o + /3)v?(-r) + ■ ■ ■ 
Following the formal adjoint theory of FDEs [25], let the phase space C be decomposed according to A as 
C = P Q)Q, where P is the center space for x = Lo(xt), i.e., P is the generalized eigenspace associated 
with A. Consider the bilinear form (•, •) associated with the linear equation x = Lo(xt) [23]. Let $ and 
be bases for P and P* associated with the eigenvalues ±zo>o of the adjoint equation, respectively, and 
normalize them so that ($, = I. In complex coordinates, are written as 2 x 2 matrices of the 
form 



Ms) 



*(*) = 



1p 2 {s) 



, Vi(s) = e- l ^ s u T , V2(s)=ViO), 0<s<t 



(9) 



where the bar means complex conjugation, u T is the transpose of u, and u, v are vectors in C 2 such that 



L (<j>i) = iuj v, u T L Q (e tuJ ° s I) = iu u T , (V>i, </>i) = 1. 
Note that $ = $£>, where £? is the diagonal matrix B — diag(iw , — i^o)- From (10), we have 

v 2 = iuovu [~(b - c/'(0)) - ch f'(Q)e-^° T }v 1 = {a + iuj a )v 2 , 
Ul = (a + ilu )u 2 , [-{b - c/'(0)) - ch o f'(0)e-^ T }u 2 = ^w ^i 1 , 

Hence, we can select 



(10) 



(11) 



Vl 




1 


u = 


Ul 


= Ul 


1 


v 2 




iujQ 


u 2 


1 



(12) 



with 



Ml = 



a + iujQ 



a + 2iu - c/io/'(0)re-^ T 
Here and in the following, we refer to [22] for results and explanations of several notations involved. 
Enlarging the phase space C by considering the space BC and using the decomposition x t = &z(t)+y t , z G 
C 2 ,y t G Q 1 , we decompose system (8) as 



z = Bz + *(O)F o (&z + y,0) 

y = A Q1 y + (I-Tr)X F {$z + y,(3) 



(13) 



Consider the Taylor formula 



*(0)F (<Dz + y,/3) = -f 2 \z,y,p) + -fi(z,y, /?) + h.o.t. 

where fj(z,y,(3) (j — 2,3) are homogeneous polynomials in (z,y,(3) of degree j with coefficients in C 2 
and h.o.t. stands for higher order terms. The normal form on the 2-dimcnsional center manifold of the 
origin and (3 = is given by 



z = Bz + hg\ (z, 0, (3) + ^gl{z, 0, (3) + h.o.t. 



(14) 
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where g\,g\ are second and third order terms in (z,(3), respectively. 
Using the notations in [22], we have 



2 J " ' " ' 2 
where Proj s / is the projection of / on S, and 

Kev{Ml) = span 

After some computation, we obtain 



\g\{z,Q,(3) = 7; p ™j Kei(Mi) f2(z 7 0,f3) 




A lZl [3 



with 



A 1 = -cf'(0)e-^ T u 2 



To compute the cubic terms, we first deduce that, after the change of variables that transformed the 
quadratic terms f 2 {z,y,(3) into g 2 (z,y,p), the coefficients of the third order terms at y — 0,(3 = are 
still given by ±f£(z,0,0) (because /"(0) = 0, implying f%(z,y,0) = 0). This implies that [22] 

\gl{z,Q,P) = \?™] Kex(Ml) fl{z,Q,p) 



where 



Ker(M3) = span 




However, the terms 0(\z\(3 2 ) are irrelevant to the determination of the generic Hopf bifurcation. Hence, 
we write 



for 



Some computations yield 



with 



\g\{z, 0, 0) = hvoisfUz, 0, 0) + 0(\z\f3 2 ) 



S := span 




A 2 z\z 2 
A 2 z\z 2 



0(\z\P 2 ) 



cf'"(0) 

A 2 = -^^(1 - h Q e-^> T )u 2 



Thus, we obtain the normal form (14) with the coefficients A\,A 2 given explicitly in terms of the 
original equation (4), without the need to compute the center manifold beforehand. The normal form 
(14) can be written in real coordinates (x, y), through the change of variables z\ = x — iy, z 2 = x + iy. 
In polar coordinates (r, 6), x = rcos0, y = rsin#, this normal form becomes 



r = Krfr + K 2 r 3 + 0((3 2 r + |(r, /?)| 4 ) 
e = -w + O(\(r,(3)\) 



(15) 
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where K\ := ReAi,K 2 := ReA 2 . 

We have the following theorem. 
Theorem 3: In formula (15), the sign of K\K 2 determines the direction of the Hopf bifurcation: if 
K\K 2 < (> 0), then the Hopf bifurcation is supercritical (subcritical) and the bifurcating periodic 
solutions exist for ft > h (< ft ); the sign of K 2 determines the stability of the bifurcating periodic 
orbits: the bifurcating periodic orbits is stable (unstable) if K 2 < (> 0). 

4 A numerical example 

Consider an example in the form of system (1), with a = 1, b = 1.1, c = 1, r = 1, and /(.) = tanh(.). 
The theoretical analysis in Section 2 leads to 

loo = 0.9017, ho = 1.1496 

It then follows from the results in Section 3 that 

K x = 0.2627, K 2 = -0.3408 

These calculations prove that the equilibrium (0, 0) is stable when h < ho, as shown by Fig. 1, where 
h = 1.1. When h passes through the critical value h = 1.1496, the equilibrium losses its stability 
and a Hopf bifurcation occurs, i.e., a family of periodic solutions bifurcate from the equilibrium. Each 
individual periodic orbit is stable for K 2 < 0. Since K\K 2 < 0, the bifurcating periodic solutions exist at 
least for values of h slightly larger than the critical value. Choosing h = 1.4, as predicted by the theory, 
Fig. 2 shows that there is an orbitally stable limit cycle. 




-0.2 -0.1 0.1 0.2 0.3 20 40 60 80 100 



Figure 1: Phase plot and waveform plot for system (1) with ft = 1.1 

5 Chaotic behavior 

In this section, we study the dynamical behavior of system (1) with the activation function f(x) = 
xe -x 2 /2 an( j a _ q g = 1 3 c = 5, t = 5, and let ft be a variable parameter. It is noted that several 
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-1 -0.5 0.5 1 20 40 60 80 100 



x/t) t 

Figure 2: Phase plot and waveform plot for system (1) with h = 1.4 

other parameters have also been examined and shown to exhibit similar dynamical phenomena. Due to 
limitation of space, those results are not presented here. 

When h < 0.65, the system is stable. When increasing h to h = 0.7, the system produces a periodic 
orbit. When h = 0.7, the phase plot and the waveform plot of X\{t) is shown in Fig. 3. When the value 
of h passes 0.9, the system becomes chaotic. In Fig. 4, we show the phase plot when h = 1.0, and in 
Fig. 5 we show the waveform of x\{t) and the power spectrum plots when h = 1.0. When h = 5.0, the 
phase plot and the waveform of x\{t), and the power spectrum plots, are shown in Fig. 6 and Fig. 7, 
respectively. From these figures, we can see that the system is also chaotic, but it is different from the 
case of h = 1.0. 




-2 1 1 1 1 1 1 1 -3 1 1 1 1 1 

-3 -2 -1 1 2 3 50 100 150 200 



x,(t) t 

Figure 3: Phase plot and waveform plot when h = 0.7 

6 Conclusions 

A single delayed neuron model with inertial term has been investigated in this paper. Hopf bifurcation is 
studied by using the normal form theory of retarded FDEs, in which the coefficients of the normal form 
are obtained in terms of the original delayed equation directly, without the need to compute the center 
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100 200 300 400 500 0.02 0.04 0.06 0.08 0.1 

t V 

Figure 5: Waveform and power spectrum plots when h = 1.0 

manifold beforehand, which simplifies the computation procedure. With a nonmonotonic activation 
function, chaotic behavior has also been observed in this system. 
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10 1 1 1 1 1 1 

-15 -10 -5 5 10 15 

x,(t) 

Figure 6: Phase plot when h = 5.0 




100 200 300 400 500 0.05 0.1 0.15 0.2 0.25 0.3 

t v 

Figure 7: Waveform and power spectrum plots when h — 5.0 
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